H-Pastry: An Adaptive Multi-level Overlay Inter-Network

Overlay networks are widely used for locating and disseminating information, by means of custom routing and forwarding on top of the underlying network. Distributed Hash Table (DHT) based overlays provide flexibility coupled with good scalability and load balancing properties. However, this comes at the cost of inefficient routing, caused by the lack of adaptation to the underlying network: current DHT designs often overlook physical network proximity, administrative boundaries and inter-domain routing policies, or a combination thereof. In this paper we show how to construct a DHT-based overlay network that takes all these aspects into account. Based on the Pastry distributed object location and routing substrate and the Canon paradigm, we developed HPastry, an overlay DHT scheme that harvests the scalability and load balancing features of flat DHT designs, while also adapting to the underlying network topology, administrative structure and routing policies. We evaluate the performance characteristics of the proposed scheme through an extensive set of detailed simulations over realistic inter-network topologies. Our results show that H-Pastry substantially improves routing by reducing both overlay path stretch (by up to 50%) and routing policy violations (by up to 67%), compared to the canonical Chord DHT. In addition, the canonical design of HPastry keeps traffic within administrative boundaries as far as possible, while also creating excellent opportunities for the support of caching and multicast. Keywords-overlay; hierarchical; DHT; Pastry;